Presenting personalized advertisements in a movie theater based on emotion of a viewer

ABSTRACT

A method for presenting advertisements based on emotion of a viewer is provided. A first emotional state of a viewer wearing smart glasses is detected using first sensor data captured by a sensor of the glasses. A rate is set according to the detected first emotional state. A set of advertisements is displayed on the smart glasses according to the set rate while a movie is played so that the advertisements are perceived by the viewer as overlaying the movie.

BACKGROUND

The present invention relates to presenting personalized advertisements,and more particularly, to presenting personalized advertisements in amovie theater based on emotion of a viewer.

Generally, three-dimensional (3D) movies displayed in a movie theaterrequire a viewer to wear glasses to experience the movie in 3D.Typically, an admission ticket to a 3D movie costs more than anadmission ticket to a two-dimensional (2D) movie. Accordingly, a movietheater might consider making 3D movies more affordable to movie-goersby incentivizing the movie-goers to view advertisements (Ads) while the3D movie is playing. For example, the movie theater may offer a reducedcost ticket to a 3D movie-goer who agrees to watch Ads during the movie.However, some movie-goers might opt for a non-incentivized 3D movieexperience. Thus, no Ads will be displayed to them. Accordingly, for thesame movie performance, Ads may need to be shown to some movie-goers butnot to all movie-goers.

Thus, there is a need to accommodate both types of movie-goers duringthe same movie performance.

SUMMARY

According to an exemplary embodiment of the present invention, a methodfor presenting advertisements based on emotion of a viewer includesdetecting a first emotional state of a viewer wearing smart glassesusing first sensor data captured by a sensor of the glasses. A rate isset according to the detected first emotional state. A set ofadvertisements is displayed on the smart glasses according to the setrate while a movie is played so that the advertisements are perceived bythe viewer as overlaying the movie.

According to an exemplary embodiment of the present invention, anapparatus for presenting advertisements based on emotions of a viewerincludes a frame including a display glass, an eye movement sensor, amemory storing a computer program, and a processor configured to executethe computer program. The computer program is configured to detect afirst emotional state of a viewer wearing the frame using first sensordata captured by the sensor, set a rate according to the detected firstemotional state, and display a set of advertisements on the displayglass according to the set rate while a movie is played so that theadvertisements are perceived by the viewer as overlaying the movie.

According to an exemplary embodiment of the present invention, a systemfor presenting advertisements based on emotions of a viewer includessmart glasses configured to detect a first emotional state of a viewerusing first sensor data captured by a sensor, set a rate according tothe detected first emotional state, and display a subset ofadvertisements according to the set rate while a movie is played so thatthe advertisements are perceived by the viewer as overlaying the movie.A server is configured to send a set of advertisements comprising thesubset to the smart glasses.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

Through the more detailed description of some embodiments of the presentdisclosure in the accompanying drawings, features of the presentdisclosure will become more apparent, wherein:

FIG. 1 illustrates a cloud computing environment according to anembodiment of the present invention;

FIG. 2 depicts abstraction model layers according to an embodiment ofthe present invention, which may be used to implement a validatorprocessing unit;

FIG. 3 shows an exemplary computer system/server, which is applicable toimplementing embodiments of the present invention;

FIG. 4 shows a system for presenting advertisements based on emotion ofa viewer according to embodiments of the present invention;

FIG. 5 illustrates a movie frame and an advertisement according toembodiments of the present invention;

FIGS. 6A and 6B illustrate a method for using the system for presentingadvertisements based on emotion of a viewer according to embodiments ofthe present invention; and

FIG. 7 illustrates a system used to select a set of advertisements basedon movie metadata according to embodiments of the present invention.

DETAILED DESCRIPTION

It is understood in advance that although this disclosure includes adetailed description on cloud computing, implementation of the teachingsrecited herein are not limited to a cloud computing environment. Rather,embodiments of the present invention are capable of being implemented inconjunction with any other type of computing environment now known orlater developed.

Cloud computing is a model of service delivery for enabling convenient,on-demand network access to a shared pool of configurable computingresources (e.g. networks, network bandwidth, servers, processing,memory, storage, applications, virtual machines, and services) that canbe rapidly provisioned and released with minimal management effort orinteraction with a provider of the service. This cloud model may includeat least five characteristics, at least three service models, and atleast four deployment models.

Characteristics are as follows:

On-demand self-service: a cloud consumer can unilaterally provisioncomputing capabilities, such as server time and network storage, asneeded automatically without requiring human interaction with theservice's provider. Broad network access: capabilities are availableover a network and accessed through standard mechanisms that promote useby heterogeneous thin or thick client platforms (e.g., mobile phones,laptops, and PDAs).

Resource pooling: the provider's computing resources are pooled to servemultiple consumers using a multi-tenant model, with different physicaland virtual resources dynamically assigned and reassigned according todemand. There is a sense of location independence in that the consumergenerally has no control or knowledge over the exact location of theprovided resources but may be able to specify location at a higher levelof abstraction (e.g., country, state, or datacenter).

Rapid elasticity: capabilities can be rapidly and elasticallyprovisioned, in some cases automatically, to quickly scale out andrapidly released to quickly scale in. To the consumer, the capabilitiesavailable for provisioning often appear to be unlimited and can bepurchased in any quantity at any time.

Measured service: cloud systems automatically control and optimizeresource use by leveraging a metering capability at some level ofabstraction appropriate to the type of service (e.g., storage,processing, bandwidth, and active user accounts). Resource usage can bemonitored, controlled, and reported providing transparency for both theprovider and consumer of the utilized service.

Service Models are as follows:

Software as a Service (SaaS): the capability provided to the consumer isto use the provider's applications running on a cloud infrastructure.The applications are accessible from various client devices through athin client interface such as a web browser (e.g., web-based e-mail).The consumer does not manage or control the underlying cloudinfrastructure including network, servers, operating systems, storage,or even individual application capabilities, with the possible exceptionof limited user-specific application configuration settings.

Platform as a Service (PaaS): the capability provided to the consumer isto deploy onto the cloud infrastructure consumer-created or acquiredapplications created using programming languages and tools supported bythe provider. The consumer does not manage or control the underlyingcloud infrastructure including networks, servers, operating systems, orstorage, but has control over the deployed applications and possiblyapplication hosting environment configurations.

Infrastructure as a Service (IaaS): the capability provided to theconsumer is to provision processing, storage, networks, and otherfundamental computing resources where the consumer is able to deploy andrun arbitrary software, which can include operating systems andapplications. The consumer does not manage or control the underlyingcloud infrastructure but has control over operating systems, storage,deployed applications, and possibly limited control of select networkingcomponents (e.g., host firewalls).

Deployment Models are as follows:

Private cloud: the cloud infrastructure is operated solely for anorganization. It may be managed by the organization or a third party andmay exist on-premises or off-premises.

Community cloud: the cloud infrastructure is shared by severalorganizations and supports a specific community that has shared concerns(e.g., mission, security requirements, policy, and complianceconsiderations). It may be managed by the organizations or a third partyand may exist on-premises or off-premises.

Public cloud: the cloud infrastructure is made available to the generalpublic or a large industry group and is owned by an organization sellingcloud services.

Hybrid cloud: the cloud infrastructure is a composition of two or moreclouds (private, community, or public) that remain unique entities butare bound together by standardized or proprietary technology thatenables data and application portability (e.g., cloud bursting forload-balancing between clouds). A cloud computing environment is serviceoriented with a focus on statelessness, low coupling, modularity, andsemantic interoperability. At the heart of cloud computing is aninfrastructure comprising a network of interconnected nodes. Referringnow to FIG. 1, illustrative cloud computing environment 50 is depicted.As shown, cloud computing environment 50 comprises one or more cloudcomputing nodes 10 with which local computing devices used by cloudconsumers, such as, for example, personal digital assistant (PDA) orcellular telephone 54A, desktop computer 54B, laptop computer 54C,and/or automobile computer system 54N may communicate. Nodes 10 maycommunicate with one another. They may be grouped (not shown) physicallyor virtually, in one or more networks, such as Private, Community,Public, or Hybrid clouds as described hereinabove, or a combinationthereof. This allows cloud computing environment 50 to offerinfrastructure, platforms and/or software as services for which a cloudconsumer does not need to maintain resources on a local computingdevice. It is understood that the types of computing devices 54A-N shownin FIG. 1 are intended to be illustrative only and that computing nodes10 and cloud computing environment 50 can communicate with any type ofcomputerized device over any type of network and/or network addressableconnection (e.g., using a web browser).

Referring now to FIG. 2, a set of functional abstraction layers providedby cloud computing environment 50 (FIG. 1) is shown. It should beunderstood in advance that the components, layers, and functions shownin FIG. 2 are intended to be illustrative only and embodiments of theinvention are not limited thereto. As depicted, the following layers andcorresponding functions are provided:

Hardware and software layer 60 includes hardware and softwarecomponents. Examples of hardware components include: mainframes 61; RISC(Reduced Instruction Set Computer) architecture based servers 62;servers 63; blade servers 64; storage devices 65; and networks andnetworking components 66. In some embodiments, software componentsinclude network application server software 67 and database software 68.

Virtualization layer 70 provides an abstraction layer from which thefollowing examples of virtual entities may be provided: virtual servers71; virtual storage 72; virtual networks 73, including virtual privatenetworks; virtual applications and operating systems 74; and virtualclients 75.

In one example, management layer 80 may provide the functions describedbelow. Resource provisioning 81 provides dynamic procurement ofcomputing resources and other resources that are utilized to performtasks within the cloud computing environment. Metering and Pricing 82provide cost tracking as resources are utilized within the cloudcomputing environment, and billing or invoicing for consumption of theseresources. In one example, these resources may comprise applicationsoftware licenses. Security provides identity verification for cloudconsumers and tasks, as well as protection for data and other resources.User portal 83 provides access to the cloud computing environment forconsumers and system administrators. Service level management 84provides cloud computing resource allocation and management such thatrequired service levels are met. Service Level Agreement (SLA) planningand fulfillment 85 provide pre-arrangement for, and procurement of,cloud computing resources for which a future requirement is anticipatedin accordance with an SLA.

Workloads layer 90 provides examples of functionality for which thecloud computing environment may be utilized. Examples of workloads andfunctions which may be provided from this layer include: mapping andnavigation 91; software development and lifecycle management 92; virtualclassroom education delivery 93; data analytics processing 94;transaction processing 95; and an advertisement processing unit 96. Thefunctions performed by server 220, which will be discussed below in moredetail with respect to FIG. 4-7 may be performed by the advertisingprocessing unit 96.

The inventive concept will be described in more detail with reference tothe accompanying drawings, where exemplary embodiments of the presentdisclosure have been illustrated. Throughout the drawings, same or likereference numerals are used to represent the same or like components.However, the present inventive concept can be implemented in variousmanners, and thus should not be construed to be limited to theembodiments disclosed herein. On the contrary, those embodiments areprovided for the thorough and complete understanding of the presentdisclosure to convey the scope of the present disclosure to thoseskilled in the art.

A movie theater that shows three-dimensional (3D) movies may offermovie-goers an option to watch advertisements (Ads) during the screeningof the 3D movie to reduce the 3D movie admission cost. Each viewed Admay reduce the 3D movie admission cost by a predetermined amount.Accordingly, the more Ads a movie-goer sees during the 3D movie, themore his or her admission cost will be reduced.

Generally, a movie-goer wears glasses to experience a movie in 3D. Theglasses can be smart glasses (e.g., glasses that include a computer, aglass or plastic lens (e.g., display glass) on which informationprocessed by the computer may be displayed on, sensors, microphones, andthe like) and the Ads can be displayed on the glasses as beingsuperimposed or overlaid on the movie. The 3D movie is displayed on amovie theater screen and perceived by the movie-goer in 3D through thesmart glasses.

When the movie-goer chooses to watch Ads during a 3D movie, a set of Adsto be displayed during the movie is selected based on the type of moviechosen. Each Ad is mapped (e.g., linked) to a particular objectdisplayed on a particular movie frame. A movie frame may be a stillimage component of a movie.

For a different 3D movie experience, an Ad frequency corresponding to anumber of Ads displayed over a predetermined amount of time during themovie is selected based on the movie-goer's interest in the movie. Inaddition, each Ad is displayed on an uninteresting part of a frame. TheAd frequency may be changed during the movie according to themovie-goer's emotional state. In addition, a movie-goer may have theoption to stop the Ads. Different movie-goers watching the same 3D moviemay see different Ads at different frequencies and different times.

FIG. 3 illustrates an embodiment of a computer server that may be usedto implement part of computing devices 54A-54N and advertisementprocessing unit 96, which is applicable to implementing embodiments ofthe present invention. Computer system/server 12 is only illustrativeand is not intended to suggest any limitation as to the scope of use orfunctionality of embodiments of the invention described herein.

As shown in FIG. 3, the computer system/server 12 is shown in the formof a general-purpose computing device. The components of the computersystem/server 12 may include, but are not limited to, one or moreprocessors or processing units 16, a system memory 28, and a bus 18 thatcouples various system components including system memory 28 toprocessor 16.

Bus 18 represents one or more of any of several types of bus structures,including a memory bus or memory controller, a peripheral bus, anaccelerated graphics port, and a processor or local bus using any of avariety of bus architectures. By way of example, and not limitation,such architectures include an Industry Standard Architecture (ISA) bus,a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, aVideo Electronics Standards Association (VESA) local bus, and aPeripheral Component Interconnect (PCI) bus.

The computer system/server 12 may include a variety of computer systemreadable media. Such media may be any available media that is accessibleby the computer system/server 12, and it includes both volatile andnon-volatile media, removable and non-removable media.

The system memory 28 can include computer system readable media in theform of volatile memory, such as random access memory (RAM) 30 and/orcache memory 32. The computer system/server 12 may further include otherremovable/non-removable, volatile/non-volatile computer system storagemedia. By way of example, storage system 34 can be provided for readingfrom and writing to a non-removable, non-volatile magnetic media (notshown and typically called a “hard drive”). Although not shown, amagnetic disk drive for reading from and writing to a removable,non-volatile magnetic disk (e.g., a “floppy disk”), and an optical diskdrive for reading from or writing to a removable, non-volatile opticaldisk such as a CD-ROM, DVD-ROM or other optical media can be provided.In such instances, each can be connected to bus 18 by one or more datamedia interfaces. As will be further depicted and described below,memory 28 may include at least one program product having a set (e.g.,at least one) of program modules that are configured to carry out thefunctions of embodiments of the invention.

A program/utility 40, having a set (at least one) of program modules 42,may be stored in memory 28 by way of example, and not limitation, aswell as an operating system, one or more application programs, otherprogram modules, and program data. Each of the operating system, one ormore application programs, other program modules, and program data orsome combination thereof, may include an implementation of a networkingenvironment. The program modules 42 generally carry out the functionsand/or methodologies of embodiments of the invention as describedherein.

The computer system/server 12 may also communicate with one or moreexternal devices 14 such as a keyboard, a pointing device, a display 24,etc.; one or more devices that enable a user to interact with thecomputer system/server 12; and/or any devices (e.g., network card,modem, etc.) that enable the computer system/server 12 to communicatewith one or more other computing devices. Such communication can occurvia Input/Output (I/O) interfaces 22. The computer system/server 12 cancommunicate with one or more networks such as a local area network(LAN), a general wide area network (WAN), and/or a public network (e.g.,the Internet) via a network adapter 20. As depicted, the network adapter20 communicates with the other components of computer system/server 12via the bus 18. It should be understood that although not shown, otherhardware and/or software components could be used in conjunction withthe computer system/server 12. Examples of these other hardware and/orsoftware components include, but are not limited to: microcode, devicedrivers, redundant processing units, external disk drive arrays, RAIDsystems, tape drives, and data archival storage systems.

FIG. 4 shows a system for presenting advertisements based on emotion ofa viewer according to an exemplary embodiment of the invention.Referring to FIG. 4, the system includes smart glasses 210, a mobiledevice 250, a server 220, and Ad sources 240.

The Ad sources 240 may include a plurality of stores 241 and outsidesources 242. The stores 241 may be included in a mall that includes amovie theater that displays 3D movies. The stores 241 may include, forexample, an apparel store, a home improvement store, a shoe store, acandy store, and the like. Each of the stores 241 may have one or moreAds featuring their products.

An Ad may be an image of a product or a video of a product. According toan embodiment, an Ad is a still image of a product. For example, theshoe store may have a first Ad featuring a rugged hiking shoe and asecond Ad featuring a stiletto shoe. The outside sources 242 may includecommercial entities that do not include establishments (e.g., stores)located in the mall, each of which having its own Ads featuring itsproducts.

The Ads of the stores 241 and outside sources 242 are transferred to aserver 220. Each Ad of the Ad sources 240 may include metadata regardingthe objects displayed in the Ad. For example, the shoe store Adfeaturing the stiletto shoe may include metadata describing the stilettoshoe as a woman's shoe, a stiletto shoe, a heeled-shoe, a formal shoe, afashionable shoe, and the like.

The server 220 may store the Ads of the Ad sources 240 and may select aset of Ads, from among the stored Ads, to display to a viewer of a 3Dmovie based on a movie input 230. The server 220 may be a computersimilar to the computer described above with reference to FIG. 7. Theserver 220 may be disposed in the mall where the movie theaterdisplaying the 3D movie is located, inside the movie theater, in a safelocation outside of the mall, or in the cloud.

The movie input 230 includes a 3D movie that a movie-goer has chosen towatch. In addition, the movie input 230 includes metadata regardingobjects displayed in the 3D movie (e.g., on different frames of themovie). The movie input 230 may include metadata describing a type ofthe 3D movie, for example, an action movie, a drama, a comedy, or thelike. The movie input 230 may include metadata about each frame of theselected 3D movie, describing, for example, each object displayed oneach frame of the movie and interesting and uninteresting areas of eachframe.

The server 220 may select the set of Ads to be displayed during aparticular movie based on the metadata of the selected 3D movie and themetadata of the Ads of the Ad sources 240. For example, the server 220may use classifiers to match the metadata of the selected movie with themetadata of the Ads received from the Ad sources 240. The server 220 maytransfer the set of Ads to be displayed during a particular movie to acomputer 211 of a wearable glass 210.

A mobile device 250 may be a handheld mobile device, a wireless phone, atablet computer, or the like, in the possession of the movie-goer. Themobile device 250 may be communicatively coupled to the computer 211 ofa wearable glass 210 using, for example, standard protocols. The mobiledevice 250 may include a user profile 251 and an Ad store 252. The userprofile 251 may include personal information about the movie-goer suchas the movie-goer's name, address, age, and other information such asthe movie-goer's interests, hobbies, personal preferences, and the like.The Ad store 252 may include a plurality of Ads that the movie-goer maywatch during his or her leisure time in the mobile device 250. These Adsmay be input to the Ad store 252 by the computer 211. The Ad store 252stores the Ads that a user (e.g., movie-goer) likes (e.g., determinedbased on gaze, winks, etc.)

The wearable glass 210 may be worn on the head of the movie-goer. Thewearable glass 210 may include one or more sensors 212, the computer211, a display glass 213 attached to a frame of the wearable glass 211,a microphone, and speakers that may be disposed on the movie-goer'sears. The display glass 213 may be a glass or plastic lens disposed infront on the movie-goer's eyes. The display glass 213 may display the 3Dmovie and the Ads selected by the server 220. The display glass 213 maydisplay the Ads as being superimposed on the 3D movie. For example,projector may be used to project a 3D movie on a movie theater screenfor a plurality of movie-goers to watch. Each movie-goer wears his/hercorresponding wearable glass 210. Each movie-goer perceives the 3D moviein 3D through the display glass 213. The Ads are displayed on thedisplay glass 213 as superimposed on the 3D movie, for each movie-goer,from among the plurality of movie goers watching the same movie.

The computer 211 selects which Ads, from among the plurality of Adsselected by the server 220, to be displayed during the 3D movie, when todisplay the selected Ads as the movie is playing, and where to locatethe Ads in each movie frame. For example, an Ad may be displayed in anuninteresting area of a frame, which may correspond to an uninterestingbackground, or to an area away from an interesting area of a particularmovie frame, using the movie input 230 metadata. Virtual coordinates ofthe uninteresting areas of each frame may be included in the metadataincluded in the movie input 230. Uninteresting parts of a frame areparts of the frame where the system discovers few image features. Forexample, the image features may be detected using scale-intervariantfeature transform (SIFT) and speeded up robust features (SURF) or evendeep learning based object recognition techniques.

The sensors 212 of the wearable glass 210 may include sensors thatcapture input data that the computer 211 can use to determine themovie-goer's emotional state. The sensors 212 include sensors thatdetect facial muscle, eyeball, and eyebrow muscle position and movement.The input data of the sensors 212 includes the detected facial muscle,eyebrow muscle, and eyeball, and position. In addition, the input dataof the sensors 212 may include movement of the eyeballs, eyebrows, andfacial muscles of the movie-goer. The sensors 212 may also include acamera disposed on the frame or glass of the wearable glass 210, thecamera being pointed to the movie-goer's face. The camera may captureimages or video of the movie-goer's face. The input data of the sensors212 includes the captured images or video of the movie-goer's face. Thesensors 212 may continuously monitor the movie-goer (e.g., continuouslydetect facial muscle, eyebrow muscle, and eyeball position and movementas well as continuously capture images or videos of the movie-goer'sface).

The computer 211 determines a first emotional state of the movie-goerusing the input data obtained by the sensors 212. The computer 211 setsa first Ad display rate based on the first emotional state of themovie-goer. For example, the emotional state of the movie-goer may bedetermined to be that the movie-goer is interested or dis-interested inthe movie when monitored by the sensors 212. When the movie-goer isinterested in the movie, the first Ad display rate may be set to zero orto a number that is smaller than the first Ad display rate set when themovie-goer is disinterested in the movie.

When the movie-goer is interested in the movie, the first Ad displayrate may be, for example, one Ad every three minutes. When themovie-goer is disinterested in the movie, the first Ad display rate maybe, for example, three Ads every three minutes. The computer 211 maydetermine a second emotional state of the movie-goer at later period intime (e.g., after the first emotional state). The second Ad display ratemay remain the same as that set based on the first emotional state, orthe second Ad display rate may be decreased, depending on the secondemotional state of the movie-goer. For example, if the second emotionalstate of the movie-goer is determined to be positive or non-negative(e.g., that the movie-goer shows a positive emotion or a non-negativeemotion to a displayed Ad), the first Ad display rate may be maintained.If the second emotional state of the movie-goer is determined to benegative (e.g., that the movie-goer shows a negative emotion in responseto a displayed Ad), the first Ad display rate may be decreased. Forexample, the Ad display rate may be adjusted depending on themovie-goer's emotional response to the displayed Ads.

The computer 211 may continuously adjust the Ad display rate using thecontinuously determined emotional states of the movie-goer. In addition,the movie-goer may have the option to turn off the Ads.

According to an embodiment, the system of FIG. 4 may include a pluralityof wearable glasses 210. The system of FIG. 4 may also include aplurality of mobile devices 250. Each wearable glass 210 of theplurality of wearable glasses 210 may be communicatively coupled to acorresponding mobile device 250. Each computer 211 of a wearable glass210, from among the wearable glasses 210, may be connected to the server220, and to an Ad store 252 and user profile 251 of a correspondingmobile device 250. A display glass 213 of a wearable glass 210 maydisplay different Ads at different times from those displayed by anotherdisplay glass 213 of another wearable glass 210. Accordingly, differentmovie-goers watching the same 3D movie in the same movie theater roommay view different Ads, at different times and different frequencies(e.g., Ad display rate).

According to an embodiment, the system of FIG. 4 is used to display Adsto movie-goers watching a two-dimensional (2D) movie. All aspects of thepresent invention relating to displaying Ads during a 3D movie apply todisplaying Ads during a 2D movie except for the movie not beingperceived in 3D by movie-goers. The 2D movie is shown on a movie theaterscreen. The Ads are displayed on the display glass 213 as superimposedon the 2D movie, for each movie-goer, from among a plurality of moviegoers watching the same 2D movie. Different movie-goers watching thesame 2D movie in the same movie theater room may view different Ads, atdifferent times and different frequencies.

FIG. 5 illustrates a movie frame and an advertisement according to anexemplary embodiment of the invention.

Referring to FIG. 5, a movie frame may correspond to a still image ofthe 3D movie. The frame may display a background, which may correspond,for example, to a mountain. An object may be displayed on the frame. Theobject may be, for example, a car tire. For ease of illustration, a caron which the tire may be mounted is not shown in FIG. 5. The Ad displaysa car tire overlaid on the movie frame. Accordingly, the Ad is of a sameor similar nature as that of the object displayed in the frame. The cartire displayed in the Ad may correspond to a car tire sold by one of thestores in the mall (e.g., a store from among the stores 241 of FIG. 4).The 3D movie frame illustrated in FIG. 5 may be displayed on thewearable glass 210 illustrated in FIG. 5.

FIGS. 6A and 6B illustrate a method for using the system for presentingadvertisements based on emotion of a viewer according to embodiments ofthe present invention.

According to an embodiment, the system for presenting advertisementsbased on emotion of a viewer, illustrated with reference to FIG. 4, canbe used to perform the method steps of the method for using the systemfor presenting advertisements based on emotion of a viewer, illustratedwith reference to FIGS. 6A and 6B.

Ads are input to a server in step S410. The server may correspond to theserver 220 of FIG. 4. The Ads may originate from stores in a shoppingmall in which a movie theater is located. In an embodiment, Ads mayoriginate from sources which exclude the stores in the mall, forexample, online retailers, restaurants not located in the mall, and thelike. In an embodiment, Ads originating from stores in the shopping malland Ads originating from sources which exclude the stores in the mallmay be input to the server in step S410.

Each Ad input to the server in step S410 may include metadata describingobjects displayed or featured in that Ad. For example, an automotivetire shop located in the mall may have an Ad featuring rugged, oroff-road tires. The Ad may include, for example, an image of a rugged,or off-road tire. The metadata of this Ad may include keywords such as“tires,” “rugged tires,” “off-road tires,” “performance tires,” and thelike.

The movie theater located in the above-mentioned shopping mall maydisplay various 3D movies at various time periods. Each 3D moviedisplayed in the movie theater may include metadata about the contentsof the movie. The metadata of the movie may include an annotation ordescription of objects displayed in the movie, the background, roads,buildings, waterways, and the like, displayed in the movie. Each 3Dmovie may include metadata about persons displayed in the movie, forexample, metadata describing the actors, the articles of clothing wornby the actors, jewelry worn by the actors, and the like. In addition,each frame of each movie may be annotated with the metadata as describedabove. For example, in “Movie C,” frame 1234, may be annotated withmetadata describing the articles of clothing that an actor is wearingand that the actor is, for example, driving in a jungle. In “Movie C,”the metadata of frame 1234 may also include, for example, a descriptionof the background, objects located in the background, a virtualcoordinate (e.g., coordinate) where an object is located in the 1234frame, the articles of clothing that the actor is wearing, anidentification of the actors displayed in that frame, coordinates of anuninteresting area of the frame, and coordinates of an interesting areaof the frame. The above or alike metadata may be created for each frameof each movie displayed in the movie theater. The metadata may alsoinclude coordinates of each object and person shown in each frame.However, it is understood that the metadata of each frame is not limitedto the above items.

A user may chose to watch a 3D movie in the above-mentioned shoppingmall. For example, the user elects to watch “Movie C.” The 3D movie thatthe user chooses to watch along with its metadata is input to the serverin step S420.

The user may possess a mobile device on his/her person. The mobiledevice may include a user profile indicating personal data of the usersuch as the user's name, address, age, gender, and the like. Inaddition, the user profile may include information such as objects thatthe user likes, owns, or would like to own, activities that the userlikes, social clubs that the user is affiliated with, and the like. Instep S430, the user profile is output from the user's mobile device andis input to the server.

In an embodiment, step S430 is optional. Accordingly, in an embodiment,the user profile is not input to the server.

In step S440, a set of Ads to be displayed during the 3D input in stepS420 are selected from among all the Ads input to the server in stepS410. This may be done by a classifier/matching engine running in theserver that matches the metadata of the input Ads with the metadata ofthe movie chosen by the user. The classifier/matching engine may selectthe set of Ads to be displayed during the selected movie, for example,by comparing and matching the metadata of the selected movie with themetadata of the input Ads.

In addition, in step S440, the set of Ads may also be created based on aclassification of the chosen movie and the classification of the inputAds. The classification of a movie may include a movie genre. Forexample, some Ads input in step S410 may be classified as Ads featuringsports gear. A 3D movie chosen in step S420 may be classified as anaction movie. A match may be made between an action movie and sportsgear. For example, actors in the action movie may wear sports gear.Accordingly, the Ad featuring sports gear may be included in the set ofAds to be displayed when showing the action movie.

In an embodiment, a set of Ads to be displayed in each respective 3Dmovie showing in the theater, may be determined before the user selectsa movie that he or she would like to watch. In this case, in step S440,when the user chooses a 3D movie of his or her liking, the set of Ads tobe displayed during the chosen movie is known.

In an embodiment, in steps S441, the set of Ads to be displayed duringthe chosen movie may be input to the user's mobile device when the userprofile is input from the user's mobile device in step S430. Forexample, the Ads may be input to the Ad store 252 of the mobile device250 shown in FIG. 4. The user may view the Ads input to his or hermobile device at his or her convenience at a later time.

In step S443, each Ad, from among the set of Ads selected for display instep S440, may be mapped (e.g., linked or set to correspond) to eachmovie frame of the movie selection input in step S420. The objectsand/or people displayed in each frame of the movie may be identified inthe metadata of the movie.

In an embodiment, the objects and/or people displayed in each frame ofthe movie may be detected in real-time using object recognitiontechniques. When detecting objects and/or people in real-time, theobject recognition technique may include software running in the smartglasses or in the server that can detect and identify each object and/orperson displayed in each movie frame in real-time. Metadata of theidentified objects may be determined, for example, using the internet oronline services. The metadata of the Ads can be matched with themetadata of each frame or with the real-time identification of eachobject and/or person in each frame. When real-time object identificationis used, the identified object may be matched with the metadata of theset of Ads to select an Ad that matches the real-time identified object.Accordingly, Ads may be mapped to specific objects and/or personsdisplayed in each movie frame.

For example, when frame 1234 of “Movie C” shows an actor sitting in acar, the actor wearing jewelry and driving near a hiking trail, an Adfeaturing hiking shoes may be mapped to frame 1234. Accordingly, whenframe 1234 of “Movie C” is displayed on the movie-goer's smart glasses,the hiking shoe Ad may be displayed on the movie-goer's glasses assuperimposed on the image of frame 1234.

In step S450, the 3D movie selected by the user in step S420 isdisplayed on the movie theater screen and is perceived in 3D through thesmart glasses that the user is wearing.

According to an embodiment, in step S450, a 2D movie is shown on themovie theater screen and perceived in 2D through the smart glasses thatthe user is wearing. All steps of the method for using the system forpresenting Ads based on emotion of a viewer with respect to FIGS. 4A and4B apply to 2D movies. Accordingly, a plurality of users wearing smartglasses while watching the same 2D movie in a movie theater screen mayview different Ads overlaid on their respective smart glasses atdifferent frequencies and different times.

In step S460, a user's emotional state is determined using sensorsdisposed on the smart glasses. The sensors disposed on the smart glassesmay correspond to the sensors 212 of the wearable glass 210 describedabove, with reference to FIG. 4. In an embodiment, in step S460, theuser's emotional state is continuously determined in real-time.

The user's emotional state may be determined by modeling detectedmuscle, eyebrow muscle, and eyeball position and movements. In addition,the user's emotional state may be determined by analyzing images orvideos of the movie-goer's face obtained by the sensors. The user'semotional state may also be determined by interpreting facial gestures,expressions, or gaze of the user as captured in the images or video bythe sensors. The user's emotional state may indicate that the user is,for example, bored, happy, not too happy, ambivalent, excited, not tooexcited, sad, angry, nervous, fearful, or the like.

In step S470, it is determined whether the user is interested in the 3Dmovie that he or she is watching on the smart glasses. Whether the useris interested in the movie is determined based on the emotional state ofthe user identified in step S460. According to an embodiment, the useris identified as being interested in the movie when an analysis of thesensor data gathered in step S460 indicates that the user is happy orexcited. According to an embodiment, the user is identified as beingdis-interested in the movie when an analysis of the sensor data gatheredin step S460 indicates that the user is bored or has a level of interestless than a threshold amount. The threshold amount may be predetermined.

When it is determined that the user is interested in the 3D movie, afirst Ad display rate is set in step S480. According to an embodiment,the first Ad display rate is zero. No Ads are displayed to the user whenthe Ad display rate is zero. According to an embodiment, the first Addisplay rate is a positive non-zero integer. For example, the first Addisplay rate may be 4 Ads per every 5 minutes of movie time.

In step S491, the Ads mapped to the movie frames in step S443 aredisplayed on the user's smart glasses according to the first Ad rate setin step S480. In step S491 it is determined (e.g., selected) which Adswill be displayed and where, for example, the coordinates of the movieframe where the selected Ads will be displayed.

According to an embodiment, when the user profile is input in step S430,the determination of which Ads to be displayed to the user is made bytaking in consideration the information included in the user profile.For example, when the user profile indicates that the user likesjewelry, and a movie frame includes both jewelry and outdoor sportsgear, a jewelry Ad may be displayed on that frame while an outdoorsports gear Ad might not be displayed. In addition, the determination ofwhich Ads to be displayed to the user may be made by comparing themetadata of the movie associated with a detected object against themeta-data of the selected Ads.

According to an embodiment, when the user profile is not input in stepS430, an Ad to be displayed on a particular frame is determined based onpriority criteria such as, for example, a highest pay-per-display Ad(e.g., when a first Ad originator pays a first amount for displaying itsAd during the movie and a second Ad originator pays a second amountwhich is greater than the first amount for showing the second Ad, thesecond Ad will be displayed and the first Ad is not displayed). Sincethe metadata on objects on upcoming frames is known according to anembodiment, an Ad is selected to be displayed when the upcoming framewill be displayed based on priority criteria or user profile informationdata.

According to an embodiment, a plurality of Ads is displayed in the samemovie frame. In some instances, Ads are not displayed in a movie frame.This may be because there are no identifiable objects or no metadataabout objects in the frame, because the Ad display rate is zero, becausethe available Ads in the frame do not match with the user profileinformation, or because of an election to not display an available Ad ina particular frame.

In step S491, once the Ads to be displayed in a particular frame areselected, a location on the movie frame, or coordinates of the movieframe where the Ad will be displayed are determined. Frame coordinateswhere an Ad will be displayed may be determined by the metadata of theframe, which indicate what parts of the frame are uninteresting, toincrease movie-watching experience by placing Ads away from parts of theframe that the user may be interested in watching. In addition, framecoordinates for Ad displaying may be determined by using the objectrecognition techniques discussed above, which may determineuninteresting parts of the movie frame. As discussed above,uninteresting parts of a frame are parts of the frame where the systemdiscovers few image features. In an exemplary embodiment, uninterestingparts of a frame and objects and persons displayed on the frame arerecognized using the SIFT or SURF object recognition techniques. In theembodiment, an interesting part of the frame includes features that areequal to or greater than a threshold amount of features extracted usingSIFT or SURF. A non-interesting part of a frame includes a number ofextracted features that is less than the threshold amount. The inventionis not limited to use of SIFT or SURF algorithms. For example, deeplearning based object recognition techniques or other object recognitiontechniques may be used.

Accordingly, Ads may be displayed in uninteresting parts of a frame. Inan embodiment, Ads are displayed in uninteresting parts of a frame thatare near or adjacent to a corresponding reference object displayed onthe frame. For example, a car tire Ad may be disposed in anuninteresting area adjacent to a car tire shown in the movie (e.g., inthe movie frame, the car tire can be part of a wheel of a moving car).

In an embodiment, Ads are displayed in a rectangular area, and therectangular area including the Ad may be disposed in an uninterestingarea that is close or adjacent to the object of reference thatcorresponds to the Ad. For example, as shown in FIG. 5, a car tire “Ad”is disposed near a tire object of reference “Object” in an uninterestingarea of the frame. In FIG. 5, the uninteresting area of the frame maycorrespond to a blank area.

According to an embodiment, sensor data including the gaze of themovie-goer or an area of the frame that the movie-goer is looking at,determined by the gaze or eyeball tracking of the movie-goer, is used todetermine where to overlay an Ad on a frame. For example, an eyeballtracking or gaze of the user may indicate that the user is looking orfocusing on an upper left area of a movie frame. An Ad may be displayedon the upper left area of the movie frame, in an uninteresting part ofthe upper left area of the movie frame.

Accordingly, in step S491, the Ads selected for displaying may bedisplayed (e.g., overlaid) on the corresponding frames in uninterestingparts of the frame that are near or adjacent to the object of referenceof the Ad, or in uninteresting part of a portion of a frame that themovie-goer is focusing on. When two or more Ads are shown in the sameframe, each Ad may be displayed in an uninteresting area of the framenear or adjacent to the corresponding object of reference. For example,in the same frame, a car tire Ad may be displayed near a wheel of a carshown in the movie frame and a hiking shoe Ad may be displayed near ahiking trail shown in the movie frame.

In step S492 a user's emotional state may be identified using thesensors of the smart glasses as described above. In an embodiment, instep S492 the user's emotional state is identified after the first Ad isdisplayed. For example, the user's emotional state may be identifiedwhile the Ad is being displayed, immediately after the Ad has beenshown, or a short time after the Ad has been shown.

When the user's emotion identified in step S492 is positive, the Addisplay rate previously set in step S480 is maintained in step S494.Accordingly, Ads continue to be displayed to the user at the rate set instep S480. Then, the method may loop to step S491 to display additionalAds to the user at the rate set in step S480.

When the user's emotion identified in step S492 is not positive, or isnegative, the Ad display rate previously set in step S480 is decreasedin step S495. The rate may be decreased by a predetermined amount, forexample, the Ad display rate may be decreased by 1 Ad per minute. Themethod then may loop to step S491 to display Ads to the user at thereduced Ad display rate set in step S495.

According to an embodiment, after decreasing the Ad display rate in stepS495, the method may loop to step S470 to identify whether the user isinterested in the movie.

According to an embodiment, after decreasing the Ad display rate in stepS495, the user may be prompted to indicate whether he or she wishes tocontinue watching Ads or whether the user would like to turn off the Adsfor the remainder of the movie. When the user elects to turn off theAds, Ads will cease and the incentives (e.g., admission cost reduction)that the user accumulated may be determined based on the number of Adsthat the user saw. When the user elects to continue watching Ads, themethod may loop to step S470 or to step S460. The smart glasses mayinclude a physical button/slider that can be depressed or slid toindicate they wish to stop receiving Ads.

According to an embodiment, when the user watches Ads during the entirelength of the movie, the incentives that the user accumulated may bedetermined at the end of the movie based on the number of Ads that theuser saw.

According to an embodiment, when in step S470 it is determined the useris not interested in the movie, a second Ad display rate is set in stepS480. Accordingly, Ads will be displayed to the user at the second Addisplay rate. In an embodiment, the second Ad display rate is higherthan the first Ad display rate set in step S480.

After step S490, Ads may be displayed to the user according to themethod described in step S491 described above.

In step S492, the user's emotional state may be determined as describedabove.

In step S493, the user's emotion is determined as described above.

When displaying Ads at the second Ad rate set in step S490, and theuser's emotion is positive in step S493, in step S494 the second Ad rateis maintained as set in step S490 and the method may loop to display Adsto the user according to step S491.

When displaying Ads at the second Ad rate set in step S490, and theuser's emotion is negative or non-positive in step S493, the second Adrate set in step S490 is reduced in step S495, and the method may loopto step S491 or S470. In an embodiment, when displaying Ads at thesecond Ad rate set in step S490, and the user's emotion is negative ornon-positive in step S493, the user may be prompted to indicate whetherhe or she would like to stop watching Ads.

In an exemplary embodiment, when the user returns the smart glasses tothe theater at the end of the 3D movie, user profile information storedin the smart glasses is deleted. In addition, any new profileinformation inferred about the user from the smart glasses istransferred to the user's mobile device before being deleted. The newinferred profile information is transferred to the user profile 251 ofthe mobile device 250 of the system illustrated in FIG. 4.

FIG. 7 illustrates a system used to select a set of Ads based on moviemetadata according to an embodiment of the present invention. Referringto FIG. 7, the system used to select a set of Ads based on moviemetadata, according to an embodiment of the present invention, includesthe Ad sources 240, the server 220, and current running movies intheater 260. The Ad sources 240 include the stores 241 and the outsidesources 242. The stores 241 include a plurality of stores Store 1 241-1,and Store 2 241-2 to Store N 241-N. N is a positive nonzero integer.

The server 220 includes an Ad database 223, a classifier/matching engine225, and a set of Ads based on movie types for each movie 227.

The current running movies in theater 260 includes a list of 3D moviesplaying in a theater. In addition, the current running movies in theater260 includes metadata about each frame of each 3D movie playing in thetheater. The current running movies in theater 260 includes the movieinput 230 of FIG. 4, which is a movie that a movie-goer chooses to view.

Each of the stores Store 1 241-1 to Store N 241-N is communicativelycoupled with the Ad database 223 and transfers Ads and the Ads'corresponding metadata to the Ad database 223. For example, each of thestores Store 1 241-1 to Store N 241-N pushes Ads and their correspondingmetadata to the Ad database 223. In addition, the outside sources 242push Ads and their corresponding metadata to the Ad database 223. Theclassifier/matching engine 225 performs a multi-label classification andtags each Ad to a set of movies based on a correlation or matchingbetween the type of movie and the type of Ad pushed to the Ad database223, using the metadata of the current running movies in theater 260 andthe Ads' metadata. Based on the classification, an Ad mapping table isgenerated for each 3D movie that is playing in the theater. The Admapping table includes a set of Ads to be played for each 3D movie thatcan be selected by a movie-goer. The Ad mapping table is output to theset of Ads based on movie types for each movie 227.

According to an embodiment, the set of Ads based on movie types for eachmovie 227 is communicatively coupled with the computer 211 of the systemof FIG. 4 and transfers to the computer 211 a set of Ads to be displayedduring a particular movie selected by a movie-goer.

The system illustrated in FIG. 7 can be implemented in the systemillustrated with reference to FIG. 4 and in the method illustrated withreference to FIGS. 6A and 6B.

The method described above may be performed in each of the plurality ofthe wearable glass 210 of the system described with reference to FIG. 4.Accordingly, different Ads may be displayed to different movie-goers atdifferent times and different frequencies, according to the method stepsdescribed with reference to FIGS. 6A and 6B. It is to be understood thata movie theater located in a shopping mall is exemplarily described. Themovie theater may be located anywhere that it is suitable, for example,in a yacht, cruise ship, aircraft, or the like. The movie theater mayalso be a stand-alone building built on a rural area or developed area.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

1. A method for presenting advertisements based on emotion of a viewer,the method comprising: detecting a first emotional state of a viewerwearing smart glasses using first sensor data captured by a sensor ofthe glasses; setting a rate according to the detected first emotionalstate; and displaying a set of advertisements on the smart glassesaccording to the set rate while a movie is played so that theadvertisements are perceived by the viewer as overlaying the movie,wherein the overlaying comprises: mapping each advertisement to a givenframe of the movie and a position within the frame, wherein the mappingcomprises: detecting an object displayed on the frame; comparingmeta-data of the movie associated with the detected object againstmeta-data of each advertisement to select one of the advertisements;determining a background image of the frame; and setting the position ofthe selected one advertisement to a first position in the frame thatcorresponds to the background image of the frame.
 2. The method of claim1, wherein the setting comprises: setting the rate to a first rate whenthe first emotional state indicates the viewer is interested; andsetting the rate to a second rate when the first emotional stateindicates the viewer is dis-interested, wherein the first rate is lessthan the second rate.
 3. The method of claim 2, further comprising:detecting a second emotional state of the viewer after theadvertisements have been overlayed using second sensor data captured bythe sensor; and determining whether the rate should be adjusted based onthe detected second emotional state.
 4. The method of claim 3, whereinthe determining comprises: decreasing the previously set rate when thedetected second emotional state corresponds to a negative emotion; andmaintaining the previously set rate when the detected second emotionalstate does not correspond to a negative emotion.
 5. The method of claim3, wherein the determining comprises: querying the viewer to indicatewhether to continue presentation of the advertisements when the detectedsecond emotional state corresponds to a negative emotion; and settingthe previously set rate to 0 when a response to the querying indicatesthat the presentation is not to continue; and maintaining the previouslyset rate when the response to the querying indicates the presentation isto continue.
 6. The method of claim 2, wherein the first emotional stateindicates the viewer is interested when an analysis of the first sensordata performed during the detecting indicates the viewer is happy orexcited.
 7. The method of claim 2, wherein the first emotional stateindicates the viewer is dis-interested when an analysis of the firstsensor data performed during detecting indicates the viewer is bored orhas a level of interest less than a threshold amount.
 8. The method ofclaim 1, wherein the overlaying further comprises: selecting the set ofthe advertisements from a larger set of advertisements based on themeta-data associated with the movie and meta-data associated with thelarger set of advertisements.
 9. (canceled)
 10. The method of claim 8,wherein the mapping further comprises: comparing meta-data within userprofile information of the viewer against the meta-data of the set ofadvertisements to select one of the advertisements.
 11. The method ofclaim 10, wherein the user profile information is downloaded from amobile device that is in communication with the smart glasses.
 12. Anapparatus for presenting advertisements based on emotions of a viewer,the apparatus comprising: a frame comprising a display glass; an eyemovement sensor; a memory storing a computer program; and a processorconfigured to execute the computer program, wherein the computer programis configured to detect a first emotional state of a viewer wearing theframe using first sensor data captured by the sensor, set a rateaccording to the detected first emotional state, and display a set ofadvertisements on the display glass according to the set rate while amovie is played so that the advertisements are perceived by the vieweras overlaying the movie, wherein the display of the set ofadvertisements includes displaying a first advertisement, from among theset of advertisements, in a first frame of the movie, wherein the firstframe of the movie is analyzed by the computer program to detect aninteresting area and an uninteresting area, wherein the interesting areaincludes more objects than the uninteresting area, and whereindisplaying the first advertisement includes placing the firstadvertisement in the uninteresting area.
 13. The apparatus of claim 12,wherein the computer program sets the rate to a first rate when thefirst emotional state indicates the viewer is interested, and sets therate to a second rate when the detected first emotional state indicatesthe viewer is dis-interested, wherein the first rate is less than thesecond rate.
 14. The apparatus of claim 12, wherein the computer programdetects a second emotional state of the viewer after the advertisementshave been overlayed using second sensor data captured by the sensor, anddetermines whether the rate should be adjusted based on the detectedsecond emotional state.
 15. The apparatus of claim 14, wherein thecomputer program decreases the previously set rate when the detectedsecond emotional state corresponds to a negative emotion, and maintainsthe previously set rate when the detected second emotional state doesnot correspond to a negative emotion.
 16. The apparatus of claim 12,further comprising a projector, wherein the computer program projectsthe movie onto the display glass using the projector.
 17. The apparatusof claim 12, wherein the sensor is a camera.
 18. A system for presentingadvertisements based on emotions of a viewer, the system comprising:smart glasses configured to detect a first emotional state of a viewerusing first sensor data captured by a sensor, set a rate according tothe detected first emotional state, and display a subset ofadvertisements according to the set rate while a movie is played so thatthe advertisements are perceived by the viewer as overlaying the movie;and a server configured to send a set of advertisements comprising thesubset to the smart glasses, wherein the display of the subset ofadvertisements includes displaying a first advertisement, from among thesubset of advertisements in a first frame of the movie, wherein thefirst frame of the movie is analyzed by the smart glasses to detect aninteresting area and an uninteresting area, wherein the interesting areaincludes more objects than the uninteresting area, and whereindisplaying the first advertisement includes placing the firstadvertisement in the uninteresting area.
 19. The system of claim 18,wherein the smart glasses displays the advertisements by selecting thesubset of the advertisements from the set of advertisements based onmeta-data associated with the movie and meta-data associated with theset of advertisements, and maps each advertisement of the selected setto a given frame of the movie and a position within the frame.
 20. Thesystem of claim 19, wherein the smart glasses are configured to receiveuser profile information from a remote mobile device, and perform themapping by comparing meta-data within the user profile informationagainst meta-data of the subset of advertisements to select one of theadvertisements, and set a position of the selected one advertisement toa position in the frame.
 21. The method of claim 1, wherein setting theposition of the selected one advertisement includes determining thefirst position in the frame and a second position in the frame, whereinthe first position corresponds to an area of the frame that includes afirst number of identified objects and the second position correspondsto an area of the frame that includes a second number of identifiedobjects, wherein the second number is greater than the first number.